Graph based over-segmentation of 3D point cloud representation of urban scenes

In the field of computer vision, over-segmentation, or super-pixels generation, has become a popular preliminary stage for high level image analysis processes (such as classification, registration, detection and recognition). New acquisition technologies, like RGBD or LiDAR cameras, enable the capturing of 3D point clouds containing both color and geometrical information. We propose a new generic approach for oversegmentation of 3D point clouds named Point Cloud Local Variation (PCLV). In the proposed approach, points are unified, following a similarity criteria of color and geometry, to form clusters named super-points, the 3D extension of superpixels. The feasibility of the proposed approach is demonstrated on scanned outdoor urban scenes. INTRODUCTION In recent years, 3D point cloud representation of geometric real-world data has gained increasing popularity in the fields of robotics, computer vision and computer aided design [1]. This is mainly due to the emergence of low cost 3D sensing devices such as the Kinect for indoor scenes and the more expensive LiDAR for outdoor scenes. Therefore, a large number of algorithms have been developed for applications such as navigation, pose estimation, segmentation, registration and surface reconstruction. One of the main challenges when working with 3D point clouds is the large number of points. This challenge becomes even greater when a real-time application is required. One of the ways to deal with the large amount of data is to process only a subset of the data. One of the open problems in the field of computer vision is semantic segmentation where the goal is to group pixels into semantical meaningful objects. In 2D segmentation, image pixels are combined to form segments, and in 3D segmentation, 3D points are combined to form segments. An efficient approach to handling the segmentation problem, both in 2D and in 3D, is to use an initial preprocessing stage of over-segmentation which allows the number of segments to be greater than the number of objects in the scene, while being substantially lower than the number of pixels or points in the original data. The main advantage of over-segmentation is that it is less constrained than high level segmentation. An over-segmentation algorithm performs well when the segment boundaries overlap with the object boundaries, while also partially covering only a single object surface area. Over-segmentation algorithms can be applied to any model represented by a point cloud, regardless of its geometrical complexity. The result would be a subdivision of the initial model into regions of specific properties. Over-segmentation may be used as either a preprocessing stage for segmentation, or as a direct alternative, since it also subdivides the data into subsets. Figure 1 illustrates a common 3D point cloud processing pipeline. Figure 1: The processing of 3D point clouds Over-segmentation using a Kmeans based approach similar to the 2D simple linear iterative clustering (SLIC) [2] was recently extended to 3D point clouds and called voxel cloud connectivity segmentation (VCCS) [3]. It has been shown to outperform existing 2D algorithms on the NYU Depth V2 dataset [4]. APPROACH In this research, we address the problem of oversegmentation in 3D point clouds, using a graphbased approach. The input of the method is a 3D point cloud representation of the boundary surface of any general object. The output is a division of the data into small segments. Figure 2 summarized the main steps of the approach: a. A graph is constructed from the input 3D point cloud data. The graph vertices are the 3D points and the edges may be constructed using Proceedings of the 34th Israeli Conference of Mechanical Engineering Faculty of Mechanical Engineering, Technion I.I.T Haifa, 21-22 November 2016 k nearest neighbors, neighbors within a radius, a triangulation or a prior knowledge. b. A descriptor is estimated for each 3D point. The descriptor is a quantity, usually a vector of several elements, which characterizes the point and its local properties. Some properties relate to geometrical characteristics or to the point's relation to its neighbors. c. A weight is assigned to each graph edge, which is the distance between two neighboring point descriptors, using a specific metric. d. The graph edges are sorted in ascending order and then merged using a decision criteria to create unions of neighboring points. These unions form the point clusters which are labeled and constitute the output. This stage is the core of the over segmentation method, and is inspired by the 2D over-segmentation algorithm, Local Variation [5]. Figure 2: Proposed approach for 3D point cloud over-segmentation RESULTS The feasibility of the approach is demonstrated on a 3D point cloud scanned by GeoSim [6] an advanced 3D city modeling of urban data company. The data was initially filtered and de-noised. The over-segmentation input 3D point cloud contained 5.75M points and was subdivided into 3500 clusters. Figure 3 depicts the input point cloud (a) and the results (b). Note in the zoomed-in section how some scene objects were segmented clearly, such as people, cars, and even sewer lids. CONCLUSIONS In this paper a new generic over-segmentation algorithm for 3D point clouds was presented. It enables the subdivision of data into locally similar subsets. The proposed algorithm may be used as a preliminary stage for semantic segmentation, or alternatively as a pre-processing stage for other high-level vision tasks. ACKNOWLEDGEMENTS This work was supported by Magnet Omek Consortium, Ministry of Industry and Trade, Israel.